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Abstract 

The formalism that allows to take into account the error of the expected 
mean background b in the statistical analysis of a Poisson process with the 
frequentistic method is presented. It is shown that the error a\, cannot be 
neglected if it is not much smaller than The resulting confidence belt is 
larger that the one for £7^ = 0, leading to larger confidence intervals for the 
mean fi of signal events. 
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I. INTRODUCTION 



The statistical treatment of experimental results obtained in a Poisson process with 
background and a small signal is difficult and controversial. Two methods are accepted by 
the Particle Data Group [Q]: the Bayesian Method and the Unified Approach, which is a 
frequentist method proposed recently by Feldman and Cousins that allows the unified 
calculation of confidence intervals and upper limits with the correct coverage (see ||). 

The Unified Approach represents a major breakthrough for a satisfactory statistical 
treatment of processes with small signals with the frequentist method. However, as already 
noted by Feldman and Cousins |2|], when the number of observed events in a Poisson process 
with mean \x is smaller than the expected background, the upper limit for \x obtained with the 
Unified Approach decreases rapidly when the background increases. Hence, by observing less 
events than the expected background an experiment can establish a very stringent upper 
bound on \i even if it is not sensitive to such small values of fi. This problem has been 
further discussed in Ref. where an alternative frequentist method has been proposed. 
This method yields confidence intervals and upper limits with all the desirable properties 
of those calculated with the Unified Approach and in addition minimizes the effect of the 
observation of less background events than expected. In the following this method will be 
called "Alternative Unified Approach". The basic features of the Unified Approach and 
the Alternative Unified Approach, which are necessary for the understanding of the present 
paper, are reviewed in Section |TJ. 

The original formulation of the Unified Approach || and of the Alternative Unified 
Approach J4j for a Poisson process with background assumed a precise knowledge of the 
expected mean background. The aim of this paper is the presentation of the extension of 
these approaches to the case in which the background is known with a non-negligible error. 
This is done in Sections |TTT| and [TV], where the probability to observe a number n of events 
in a Poisson process consisting in signal events with mean /x and background events with 
known mean b = b±Ob is derived (in Section [TTI|we consider the simpler case 07, < 6/3 and in 
Section |V|this constraint is removed), and in Section 0, where the method for deriving the 
corresponding confidence intervals in the Unified Approach and in the Alternative Unified 
Approach is presented. Conclusions are drawn in Section [VT|. 

II. POISSON PROCESSES WITH BACKGROUND 

The probability to observe a number n of events in a Poisson process consisting in signal 
events with mean fi and background events with known mean b is 

P(n\u;b) = — (u + b) n e-^ +b K (2.1) 
n\ 

The classical frequentist method for obtaining the confidence interval for the unknown pa- 
rameter /i is based on Neyman's method to construct a confidence belt ||. This confidence 
belt is the region in the n-/i plane lying between the two curves ni(/j; b, a) and ^(/x; b, a) 
such that for each value of \i 



2 



n2(/i;6,a) 

P(n E b, a), n 2 (fx; b, a)]\fx; b) = 2J P(n|//; &) = a , (2.2) 

n=ni (fi;b,a) 

where a is the desired confidence level. The two curves ni (//;&, a) and n 2 (fi;b,a) are re- 
quired to be monotonic functions of /i and can be inverted to yield the corresponding curves 
6, a) and /^(^ ft, a)- Then, if a number n D b s of events is measured, the confidence inter- 
val for fj, is [fi 2 (n \> s ; b, a), /xi(n b B ; ft, a)]. This method guarantees by construction the correct 
coverage, i.e. the fact that the resulting confidence interval [fi 2 (n ohs ;b,a), fii(n ohs ;b,a)] is 
a member of a set of confidence intervals obtained with an ensemble of experiments that 
contain the true value of \i with a probability a (in other words, 100a% of the confidence 
intervals in the set contain the true value of 

As noted by Cousins in Ref. ||, Neyman himself pointed out || that the usefulness of 
classical confidence intervals lies in the fact that the experiments in the ensemble do not 
need to be identical, but can be real, different experiments. One can see this fact in a simple 
way by considering, for example, two different experiments that measure the same quantity 
jj,. The 100a% classical confidence interval obtained from the results of each experiment 
belongs to a set of confidence intervals which can be obtained with an ensemble of identical 
experiments and contain the true value of fi with probability a. It is clear that the sum of 
these two sets of confidence intervals is still a set of confidence intervals that contain the 
true value of fi with probability a. 

In the case of a Poisson process, since n is an integer, the relation fl2.2| ) can only be ap- 
proximately satisfied and in practice the chosen acceptance intervals [ni(//; b, a), n 2 (fi; b, a)] 
are the smallest intervals such that 

P(n G [ni(fi;b,a),n 2 (fi;b,a)]\fi;b) > a. (2.3) 

This choice introduces an overcoverage for some values of /1 and the resulting confidence 
intervals are conservative. As emphasized in Ref. @, conservativeness is an undesirable 
but unavoidable property of the confidence intervals in the case of a Poisson process (it 
is undesirable because it implies a loss of power in restricting the allowed range for the 
parameter /1). 

The construction of Neyman's confidence belt is not unique, because in general there 
are many different couples of curves rii(/i; b, a) and n 2 (/i; b, a) that satisfy the relation Q2.2|) . 
Hence, an additional criterion is needed in order to define uniquely the acceptance intervals 
[ni(/i; b, a),n 2 (fj,; b, a)]. The two common choices are 

1 - a 

P(n < m(//; b, a)\fi] b) = P(n > n 2 (//; b, a)\fi; b) = — - — , (2.4) 

which leads to central confidence intervals and 

P(n < ni(fi; b, a)\/i; b) — 1 — a , (2.5) 

which leads to upper confidence limits. Central confidence intervals are appropriate for 
the statistical description of the results of experiments reporting a positive result, i.e. the 
measurement of a number of events significantly larger than the expected background. On 
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the other hand, upper confidence limits are appropriate for the statistical description of 
the results of experiments reporting a negative result, i.e. the measurement of a number of 
events compatible with the expected background. However, Feldman and Cousins ]2|] noticed 
that switching from central confidence level to upper confidence limits or vice-versa on the 
basis of the experimental data ("flip-flopping") leads to undercoverage for some values of //, 
which is a serious flaw for a frequentist method. 

Feldman and Cousins M proposed an ordering principle for the construction of the 
acceptance intervals that is based on likelihood ratios and produces an automatic transition 
from central confidence intervals to upper limits when the number of observed events in a 
Poisson process with background is of the same order or less than the expected background, 
guaranteeing the correct frequentist coverage for all values of //. The acceptance interval for 
each value of // is calculated assigning at each value of n a rank obtained from the relative 
size of the likelihood ratio 

RvA[n\n; b) = —— , 2.6 

P{n\ Hbest] 0) 

where //best = ^best{n',b) (for a fixed b) is the non- negative value of // that maximizes the 
probability P[n\\x\b\. 

Hbest{n; b) = max[0, n - b] . (2.7) 

For each fixed value of //, the rank of each value of n is assigned in order of decreasing value 
of the ratio RjjA(n\fi; b): the value of n which has bigger R\jpjji\^\ b) has rank one, the value 
of n among the remaining ones which has bigger R\jx(n\n; b) has rank two and so on. The 
acceptance interval for each value of // is calculated by adding the values of n in increasing 
order of rank until the condition Q2.3p is satisfied. 

The automatic transition from two-sided confidence intervals to upper confidence limits 
for n < b is guaranteed in the Unified Approach by the fact that //best is always non-negative. 
Indeed, since //best(^<fr; b) = 0, the rank of n < b for // = is one, implying that the interval 
< n < b for // = is guaranteed to lie in the confidence belt. 

As already noticed by Feldman and Cousins 0], when n < b the upper bound //i(n; b, a) 
decreases rather rapidly when b increases and stabilizes around a value close to 0.8 for large 
values of b. Hence, a stringent upper bound for // obtained with the Unified Approach by 
an experiment that has observed a number of events significantly smaller than the expected 
background is not due to the fact that the experiment is very sensitive to small values of //, 
but to the fact that less background events than expected have been observed. 

The Alternative Unified Approach proposed in Ref. [Q allows the construction of a clas- 
sical confidence belt which has all the desirable features of the one in the Unified Approach 
(i.e. an automatic transition with the correct coverage from two-sided confidence intervals 
to upper confidence limits when the observed number of events is of the order or less than 
the expected background) and in addition minimizes the decrease of the upper confidence 
limit //i(n;6, a) for a given n as the mean expected background b increases. The Alterna- 
tive Unified Approach is based on an ordering principle for the construction of a classical 
confidence belt that is implemented as the Feldman and Cousins ordering principle in the 
Unified Approach, but for each value of // the rank of each value of n is calculated from the 
relative size of the likelihood ratio 
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flAUA(n|/z;6)= ff^ , (2.8) 
where the reference value /i re f = // re f (n; 6) is taken to be the bayesian expected value for /x: 

AiP(Ai|n;6)d^ = n + l- £ ^ £ jfc! ' ( 2 - 9 ) 

\fe=0 ' / \fc=o / 

Here P(/j,\n; b) is the bayesian probability distribution for // calculated assuming a constant 
prior for \i > (see, for example, 0): 

1 / n lfc \ ^ 

PMn;6) = ^(/i + 6) n e-MXlT ■ ( 2 ' 10 ) 

\fe=0 / 

The assumption of a constant prior is arbitrary, but it seems to be the most natural choice if 
fi is the parameter under investigation and there is no prior knowledge on its value. Notice 
also that the arbitrariness induced by the choice of the prior is of "second order" with 
respect to the dominant arbitrariness induced by the choice of the method for constructing 
the confidence belt. 

The obvious inequality Ylk=o^b k /k\ — n ELo^ A' implies that /i re f (n; b) > 1. There- 
fore, /i re f(n; b) represents a reference value for \i that not only is non-negative, as desired in 
order to have an automatic transition from two-sided intervals to upper limits, but is even 
bigger or equal than one. This is a desirable characteristic in order to obtain a weak de- 
crease of the upper confidence limit for a given n when the expected background b increases. 
Indeed, it has been shown in Ref. [|J that for n < b the upper bound //i(n;6, a) decreases 
rather weakly when b increases and stabilizes around a value close to 1.7 for large values of b. 
This behaviour of //i(n; b, a) is more suitable for the physical interpretation of experimental 
results than the behaviour of Hi(n; b, a) in the Unified Approach. Furthermore, as shown by 
the example in Ref. ||], the upper limits fii(n;b,a) obtained with the Alternative Unified 
Approach for n < b are are in reasonable agreement with those obtained with the Bayesian 
Approach. Hence, the Alternative Unified Approach extends the approximate agreement 
between the Bayesian and frequentist methods from n S> b to n < b (although the statistical 
interpretations of the confidence intervals is different in the two methods). 



III. BACKGROUND WITH SMALL ERROR 



Let us consider an experiment that measures a Poisson process with an expected back- 
ground b = b ± 05 and a normal probability distribution function for the mean expected 
background b: 



f(b; b,a b ) 



/ Z7T (7ft 



cxp 



(b ~ bf 
2al 



(3.1) 



The importance of can be estimated by comparing it with which represents the rms 
fluctuation of the number of background events if b = b. If a b Vb the uncertainty of 
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the value of the background is much smaller than the typical fluctuation of the number of 
observed events induced by the background and can be safely neglected. Here we consider 
the possibility that a b is not much smaller than Vb and its contribution cannot be neglected. 

For simplicity, in this section we assume that a b <b/3 and we consider b varying from 
— oo to +00, neglecting the small error introduced by considering negative values of b. This 
approximation allows a simple analytic solution of all the integrals involved in the calculation. 
The general case with arbitrarily large a b and b restricted in the interval [0, +00) is treated 



in Section [V 



If // is the mean of true signal events, the probability P(n\n; b, a b ) to observe n events is 
given by 



P(n\ii;b, a b ) 



P(n\ii;b) f(b;b,a b ) db , 



(3.2) 



with the Poisson probability P(n\fj,;b) given in Eq. (|2.1|) . With the change of variable x 
(b — b + al)/a bl the probability P(n\fi; b, a b ) can be written as 



P(n\/i; b,a b ) 



nl 



fi + b 



a] 



exp 



in + b) + 



a 



21 



I n {fi, b, a b ) 



(3.3) 



where 



J„(/i, b, a b ) = (jl J ( 



k =o - > ^ + h ~°t 



(3.4) 



Here is the k th central moment of the normal distribution with unit variance, 



x k e x 12 dx . 



(3.5) 



Taking into account that J xe x2 / 2 dx = —e a ' 2//2 , the integral in Eq. (|3.5| ) can be calculated 
by parts, yielding 



A-! 



m k 



(Jfe/2)! 2 fc / 2 

for k even and m k = for k odd. From Eqs.( |3.4| ) and ( |3.6| ), we obtain 

n/2 



(3.6) 



I n (p,,b,(Tb) = 



nl 



k=0 



(n-2k)\ k\ 2 k \n + b-a 2 h 



2A- 



(3.7) 



Equation fl3.3|) gives the formula for the probability P{n\\i\ b, a b ) to observe a number n 
of events in a Poisson process consisting in signal events with mean \x and background events 
with known mean b = b ± a b , i.e. it replaces Eq. (|2.1| ) if the error a b of the calculated mean 
background is not negligible. The expression ( |3. 7| ) for I n (fi,b,a b ) is valid only if ab < b/3, 
but, as we will see in the next section, with an appropriate redefinition of I n (n,b,a b ) the 
formula (p.3|) for P(n\fi; b, a b ) is valid for any value of a b . 
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IV. BACKGROUND WITH LARGE ERROR 



In this section we present the formalism that allows treatment of cases in which o\> 
is arbitrarily large and b is restricted to the interval [0, +00). The gaussian probability 
distribution function of the mean expected background b normalized in the interval [0, +00) 
is 



f(b; b, a b ) 



N 



2-7T ab 



exp 



(6 - bf 



(b > 0) 



(4.1) 



with the normalization factor N given byQ 



N- 1 



1 + erf ( 



Vv/2 



Ob 



(4.2) 



Apart from the error function that must be evaluated numerically, the integral over db in 
Eq.(|3.2|) can still be solved analytically. Indeed, Eqs.(pT3|) and ( |3.4| ) are still valid, with 



N 
/2tt 



X 



k e~ x2 ' 2 dx. 



(4.3) 



where 



X rl 



b-af 



(4.4) 



The moments (|4.ij|) can be calculated by parts, yielding 



N 



m k 



1 + erf 



V2 



k\ 



+ 



N 



(fc/2)!2 fc / 2 ^ 



k\ 



my. 



k-2l-\ 
min 



£=0 



(k - 2£)\ 2 £ 



(4.5) 



for k even and 



N 



k-1 



(*-l)/2 2 

£=0 ^ 2 



X 



k-1l-\ 
min 



(4.6) 



for k odd. Therefore, the probability P(n\fi; b, ab) to observe a number n of events is given 
by the formula in Eq.( |3.3| ) with 



x The error function is defined by erf (x) = Jq e x<2 dx. 



7 



I n (/x, b,a b ) = — 



1 + erf 



(b 



0". 



2\ 1 



\V2 



CTb 



n/2 



\ rt: 
l^(n-2k)\k\ 2* Va* + 5-^ 



2A- 




— 



2\2 



n! fc! 



n-2A:-l)! (2fc + l)! V/x + &-<7, 



0/7 



2fc+l fc 



E 

£=0 



(A; - ^)! 



06 



2^+1 



7i ! 



£ (n-2fc)! fc! V// + 6-0-, 



2fc fe-1 

E 

£=0 



(Jfe-*)! 



(2(k-i))\ 2* \b-al 



21+1 



■ (4-7) 



These quantities have a cumbersome expression, but their numerical evaluation with a com- 
puter is not much more difficult than that of the corresponding quantities in Eq. Q3.7p (how- 
ever, the calculation of I n (/x, b, a b ) is rather difficult if 0^ > 6 because the addenda in Eq. (|4.7|) 
have alternating signs and the roundoff errors introduced by subtracting large numbers be- 
come crucial). 



V. CONFIDENCE INTERVALS 



The construction of the confidence belt for the probability Q3.3| ) follows the same pro- 
cedure described in Section |TJ but now the confidence interval for /x corresponding to a 
number n Q b s of observed events is [/i 2 (rz b s ; b, a b , a), /ii(n obs ; b, a b , a)], i.e. it depends on b 
and a b . The acceptance intervals can be constructed following the same principles discussed 
in Section [TI| and one can construct the confidence belt for central confidence intervals or 
upper confidence limits, or the confidence belt in the Unified Approach or in the Alterna- 
tive Unified Approach. This section is devoted to the presentation of the formalism for the 
implementation of the Unified Approach and of the Alternative Unified Approach. As an 
example, we will consider 6 = 3 and a b = 0, 1, 1.8. 

The quantity fii, es t(n)b, a b ) in the Unified Approach is the value of /x that maximizes 
P(n|/x; b, a b ) an d the acceptance interval for each value of /x is calculated assigning at each 
value of n a rank obtained from the relative size of the ratio 

RvA{n\n; b, a h ) = — — = -. (5.1) 

P{n\fi hest ;b, o b ) 

The value of ^estin; b,a b ) can be easily calculated by hand for n = 0,1,2, whereas for 
higher values of n it can be calculated numerically. The resulting 90% CL confidence belts 
for b = 3 and a b = 0, 1, 1.8 are plotted in Fig. [I]. We have checked that the confidence belt 
for a b < 0.2 practically coincides with the one for a b = 0, confirming the prediction that the 
contribution of a b is negligible if a b <C Vb. 

In Fig. [1|, the confidence belt for 07, = 1 has been obtained with the formulas presented 
in Section |TTTj, that are valid for a b < 6/3, whereas the confidence belt for a b = 1.8 has been 
obtained with the formulas presented in Section [TV], which are valid for any value of 07,. We 
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have checked that the confidence belt for a\, = \ calculated with the formulas presented in 
Section |^ practically coincides with the one shown in Fig. |]. 

From Fig. [I] one can see that the broadness of the confidence belt increases with 07, . This 
is due to the fact that the integral in Eq. ( |3.2|) has the effect of flattening the probability 
P(n\n\ b, as a function of n for fixed fi with respect to P(n\fi; b, crj, = 0) and this flattening 
effect increases with the size of ov The shift of the borders of the confidence belt as ab 
increases is not always monotonic because of the unavoidable overcoverage caused by the 
fact that n is an integer (see Section ||). 

The lower value of fi for which n = is out of the confidence belt in Fig. [I| is lower for 
ab = 1.8 than for ab = and ab = 1. This is caused by the fact that the ratio ( |5.1| ) for n = 
does not depend on ab- Indeed, from Eqs.( p.3|) and Q4.7|) we have 



P(n = 0|/i; b, a^ 



N 
~2 



1 + erf 



( 



a 



2\1 



\V2 



cxp 



ai 



~(n + b) + ^ 



Therefore, fibest( n = 0; b, ab) = and 



-Rua(™ = 0\fi;b,a b ) = e" 



(5.2) 



(5.3) 



On the other hand, the ratio RuA(n\fi; b, a^ for n > increases with ab because of the 
flattening of P(n|/i; b, ab) as a function of n. Hence, the rank of n = for each value of \i 
decreases with the increasing of ab, causing the peculiar behaviour of the upper bound H\{n = 
0; b, ab, a) as a function of ab exemplified in Fig. [j]. Since the possibility to set a smaller upper 
bound on fi for larger ab as a consequence of the observation of n = events is undesirable 
from the physical point of view, we think that in this case the physical interpretation of 
the experimental result should be very cautious, waiting for a better understanding of the 
background. 

In the Alternative Unified Approach the acceptance interval for each value of /i is calcu- 
lated assigning at each value of n a rank obtained from the relative size of the ratio 



RAUA(n\fi;b, a b ) 



P(n\fi;b,a b ) 
P(n\fi ie{ ;b, a b ) 



(5.4) 



where the reference value /i re f = yU re f (n; b, ab) is the bayesian expected value for fi. In order 
to calculate analytically the value of n re t{n;b,ab), it is convenient to write the probability 
Ul) as 



P(n\fj,; b, ab) = exp 



a 



21 



<V + b) + ^ 



with 



Jk{b,a b ) 



E 

£i=0 



n—k 



(n-k)\ 



■hib, a b ) 



(5.5) 



\.b-a b ) a b 
fi, (k-2j)\j\ -2> 



(5.6) 



for ab < 6/3 and 
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k / 2 n 



2^fc-2j 2j 



(fc-2j)! j! 2^ 



(2j + l)!(fc-2j-l)! < _ () 



1)! S 



2 £ (6 - a, 



2^— 1 _2^+l 



fc/2 j-1 

5^ (fc-2j)! 71 ^ 
i=o v J ' J e=o 



(j-£)\ (6 



<j. 



2\k-2£-l 



Ox. 



(5.7) 



for arbitrarily large a b . For the bayesian probability distribution function for fi with a 
constant prior, 



P(//|n; 6, a b ) 



P(n\fi;b, a b ) 
J °° P{n\n\b,a h ) dfi 



(5.8) 



one obtains 



P(p\n\ b, a b ) 



,fc=0 



n—k 



-1 



(n-Jfe)! 



(5.9) 



,fe=0 



Hence, the reference value /x re f (n; 6, cr&), which is the bayesian expected value for //, is given 
by 



£W(n; 6, at) = n + 1 - ^ fc J fe (6, cr 6 ) J ^ J k (b, a b ) 



(5.10) 



vfe=0 



,fe=0 



If a b ^Sb/3 the quantities Jfc(6, 05) are given by Eq. (|5.6|) and one can see that they are all pos- 
itive. Hence, the inequality Yl^o ^ Jk(J>, &b) < n Sa-=o ^k(b, cr 6 ) implies that fj, Te f(n; 6, a b ) > 1 
as in the case a b = (see Eq.(|2.9j) and the following discussion). On the other hand, the 
general formula (|5.7|) allows Jk(b,<J b ) to be negative and /i re f (n; 6, a b ) is not guaranteed to 
be larger than one if a b > 6/3. 

The 90% CL confidence belts in the Alternative Unified Approach for 6 = 3 and a b = 
0, 1, 1.8 are plotted in Fig. One can see again that the broadness of the confidence belt 
increases with o b . The behaviour of the upper bound H\{n = 0; 6, a b , a) as a function of a b 
is similar to the one obtained in the Unified Approach and the same caveats apply to the 
physical interpretation of the observation of n = events. 



VI. CONCLUSIONS 

We have presented the formalism that allows the error a b of the calculated mean back- 
ground 6 in the statistical analysis of a Poisson process with the frequentistic method to be 
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taken into account. This error must be taken into account if it is not much smaller than Vb, 
which represents the rms fluctuation of the number of background events. 

We have considered in particular the Unified Approach and the Alternative Unified 
Approach |JJ], that guarantee by construction a correct frequentist coverage. We have shown 
that the broadness of the classical confidence belt increases with <Jb, leading to an increase of 
the confidence intervals for the mean \i of signal events. The only exception to this behaviour 
is represented by the upper bound \i\{n = 0;b,ab,a), which decreases with the increasing 
of <7b for large values of in both approaches. Hence, the physical interpretation of the 
observation of n = events when 07, is large should be very cautious and the effort towards 
a better understanding of the background should receive high priority. 
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FIGURES 

Fig. |]. 90% CL confidence belts in the Unified Approach for b = 3 and cr^ = 0, 1, 1.8. 

Fig. 0. 90% CL confidence belts in the Alternative Unified Approach for b = 3 and 
a, = 0,1, 1.8. 
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